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JTRACT 

Reported are some of the outcomes of the second year 
a large computer assisted instruction program for drill and 
ictice in elementary school arithmetic. Conclusions drawn from an 
imination of the amount of computer assisted instruction completed 
the students and the effect of the drill and practice program cn 
2 students were that: (1) pupils were exposed to about one-third 
2 number of lessons originally intended, (2) software did not 
Dropriately ccmpensate for individual differences, (3) achievement 
St results showed no consistent pattern favoring CAI or non-CAI 
5ups, <4) the amount of drill and practice in CAI and non-CAI 
asses were not observably different, and (5) attitudes toward the 
ogram of students, teachers, administrators, and parents were 
vorable. (Author/FL) 
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TH,S DOCUMENT HAS BEEN REPRODUCED 
EXACTLY AS RECEfVEO FROM THE PERSON OR 
ORGANIZATION ORIGINATING IT. POINTS OF 
VIEW OR OPINIONS STATED DO NOT NECES- 
SARILY REPRESENT OFFICIAL OFFiCE OF EOU- 
CAT ON POSITION OR POLICY 



This report describes some outcomes of the second year of a large 
scale Computer Assisted Instruction (CAI) program for drill and practice 
in elementary arithmetic, grades 2 through 6, in New York City. 

The study concluded that; Ij pupils were exposed to approximately 
one- third the number of CAI lessons originally intended, 2) software 
did not appear to appropriately compensate for individual differences 
in ability, 3) achievement test results showed no consistent pattern 
favoring CAI or non-CAI groups, 4) apart from work at the CAI terminals, 
the amount of drill and practice in CAI and non-CAI classes were not 
observably different, and 5) attitudes toward the program of pupils, 
teachers, administrators, and parents were favorable. 

The recommendations weres 1) number of pupils on CAI should be 
controlled so that the system is not overloaded, 2) amojjnts of time 
and nuuber of exposures to the program should be varied for pupils at 
the extremes of the ability continuum, 3) coordination between class- 
room instruction end the CAI program should be improved, 4) the program 
should be modified so that data are readily retrievable for research 
and evaluation as well as management of CAI operations* 



Evaluation of the New York City Computer Assisted Instruction 
Project in Elementary Arithmetic, Second Year, 1969-70 
Theodore Abramson and Max Weiner 
The City University of New York 

In September 1968 the New York City Board of Education initiated a 
project for a large-scale test-demonstration of a computet assisted in- 
struction program for drill and practice in elementary arithmetic. This 
program was a modified version of an earlier arithmetic drill and practice 
program based on work dore by Dr. Patrick Suppes of Stanford University 
and others. The present report will describe some outcomes of the program 
in the second year. 

From the outset it must be emphasized that the present study is not 
an experiment in the classical sense. It may best be characterized as a 
descriptive study of particular groups of pupils at schools where there 
were CAl terminals compared with similar gro;/>s in schools where there were 
no CAI terminals. In the spring of 1969 the evaluation team proposed stud- 
ies to carry out a number of experiments which would help to provide more 
definitive answers to such questions as (1) to yhat extent is the learning 
attributable to CAI and not other variables, rnd (2) how do alternative 
instructional techniques compare with CAI? The proposed studies were dis- 
approved by the ESEA, Title III staff and their consultant as not feasible. 

The writers were aware of the methodological and statistical prob- 
lems Involved but also recognized that, as in any large-scale project, data 
must be collected and analyzed as appropriately as possible despite the 

Paper presented at the annual meeting of the American Educational 
Research Association, February 5, 1971, New York City. 
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lack of the desired degree of experimental control. In order to satisfy as 
many as possible of the viewpoints concerning appropriate data analysis it 
was decided to analyze the achievement test data at each grade level by using 
1) analysis of covariance on total groups, 2) analysis of variance of gains 
on total groups, and 3) analysis of variance of post- test scores on matched 
subgroups. 

Specifically stated, this evaluation examined the amount of CAI work 
completed by the pupils and the effect of the CAI drill and practice program 
on; 

1. arithmetic achievement at each grade level, from grades 2 
through grade 6. 

2. arithnetlc achievement of high and low achievers at each grade 
level, from grades 2 through 6. 

3. arith.vietic achievement of high and low achievers In grades 3 
. and 5 if the time factor is removed. 

4. the error rates end latencies of high and low achievers in 
grade 4. 

5. reading achievement of high and low achievers In arithmetic 
in grades 3 through 6, 

6. pupil opinions and attitudes toward CAI, toward arithmetic, 
and toward learning in general. 

7. teaching procedures In elementary arithmetic. 

8. the opinions and attitudes of teachers, school administrators, 
and parents. 

In addition, information was sought on a number of subsidiary ques- 
tions, namely: 



3 



Abramso’^ and Weiner 



3 



!• How is the effect of the CAI treatment Influenced by the 
pupils' sex and race? 

2. Did CAI affect learning of arithmetic concepts as measured by 
tests less familiar to pupils than the MAT? 

3. Did the 10- second time limit on CAI exercises interfere with 
learning for the pupils at the lowest achievement level? 

Conclusions 

On the basis of the information obtained during the study of the 
1969-70 New York City CAI project, the evaluation team has concluded that: 
(Tables are appended to the end of this papers ) 

1. At all grade levels the average number of concept blocks 
c<^pleted by pupils was much smaller than expected. On the 
average, approximately 55 CAI lessons were completed by a 
sampl i of 138 fourth grade pupils. The original expectation 
intended by the designers of the program was that the pupils 
would complete between 140-168 CAI lessons. 

2. With the exception of the fifth grale, there was a greater 
amount of CAI work completed by high than by low ability 
pupils despite the fact that the' five difficulty levels in^ 
the program were to have made it possible for all pupils to 
proceed through the materiel at a comparable rate. The pro- 
vision in the CAI program for matching differences in pupil 
ability to different difficulty levels was insufficient to 
lead to comparable achievement between high and low ability 
pupils as measured by the number of items completed, percent 
correct, number of time outs end Itsm latencies, (See Table 1«) 
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3« Although in general, the CAI experience as implemented in 
New York City during 1969-70 did not lead to losses, it did 
not lead to galnr in measured achievement in arithmetic 
computation beyond those gains achieved by comparison groups. 
(See Table 2.) These results contrast sharply with the re- 
sults obtained in the 1968-69 CAI evaluation. Among the pos- 
sible reasons for the difference between these results and 
those obtained in the 1968-69 CAI evaluation, two seem to be 
quite plausible. First, the effect of the CAI Innovation may 
cause an initial increase in pupil achievement when it is 
first instituted but this level of performance may not be 
subsequently maintained when the program becomes a more ac- 
4*.epted phase of the day to day school activity# Secondly, 
more CAI drill and practice exercises may have been completed 
by the pip 11s whose achievement waf measured during the first 
year of ^:he program# 

4. When the comparisons between CAI and|non-CAI groups were ex- 
amined for pupils at different grace levels, different abili- 
ties, different sexes, and different ethnic backgrounds some- 
what conflicting results were obtained# At grade five the 
significant differences favored the CAI pupils while at grade 
six the differences favored the non-CAI pupils# Other than 
these, the differences obtained were few and scattered and 
showed no consistent pattern# 

5# In general, exposure to the CAI arithmetic program could not 
be sar.d to have affected reading ability# 
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The results of an untimed test based on the work presented to 
the pupils at the CAI terminals tended to indicate that the 
program was not ideally adjusted for low ability level pupils. 
This finding corresponds to that given In statement 2. above, 
and lends further credence to last year’ s tentative finding 
based on the analysis of limited data related to this 
question. 

Although the CAI teachers reported spending more time on prep- 
aration than the non-CAI teachers, observers generally found 
little difference between CAI and non-CAI teachers in the 
length of their arithmetic lessons or the amount of time they 
devoted to drill and practice. 

Observers reported that arithmetic lessons were generally not 
well coordinated with work at the terminals but that there 
was a greater degree of coordinatior this year thaii last year. 
A large percentage of all categorieE of respondents to inter- 
views or questionnaires had a favorable attitude toward CAI 
and indicated that they would like to continue the program. 
They also felt that work at the terminals helped the children 
learn arithmetic better and the teachers indicated that this 
was especially true of the pupils in the middle ability range. 
Although the CAI and non-CAI pupils did not respond differ* 
ently about the amount of communication dealing with school - 
w«)rk that they had with their parents, the CAI parents felt 
that their children spoke to them more cfcen about school in 
general and arithmetic in particular than did the non-CAI 
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Recommendations 

Based on the above findings the evaluation team makes the following 
recommendations: 

1. The Central CAI office should have more autonomy and should be 
considered a line rather than a staff position so that crucial 
decisions related to Implementation of the CAI system can be 
uniformly carried out. This would lead to greater control over 
the number of pupils assigned to CAI so that the capability of 
the system to adequately deliver instruction to the pupils is 
not overloaded. It would also enable closer monitoring of the 
day by day operation and provide for the necessary flexibility 
in dealing with specific situations in specific schools. 

2. A concerted effort should be made to correlate the instruction 
given by the teacher and the drill and practice given at the 
CAI terminals. This would require very careful scrutiny of 
both (he New York City arithmetic curriculum and the CAI drill 
and practice program so that at the bjeginning of the school 
yecr appropriate time and sequence schedules could be drawn up 
and followed. Such scheduling My not be feasible for all the 
classes in all the schools but should be carefully done In some 
classes in some of the schools. This t>*pe of scheduling Is 
extremely Imporcant If the CAI system is to be fairly tested. 

3. those teachers who have had experience with CAI and have shown 
that they are adept at coordinating their teaching with CAI 
should work as a group to Identify their own method of relating 
their classroom practices to CAI. They should then develop a 
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guide specifically for New York City teachers, taking into 
account the curriculum and pupil characteristics, and should 
subsequently serve as demonstration teachers. 

4. Pupils at the extremes of the ability continuum should be al- 
lowed to have different amounts of time and/or niunber of ex- . 
posures to the system. Schedules might be arranged so that 
the most able pupils might have 20-25 minutes of CAI a week, 
whereas those at the other extreme might be given as long as 
80-100 minutes. 

5. Questions pertaining to behaviors and attitudes of those in- 
volved in the CAI program (pupils, teachers, parents) should 
continue to be examined by the school administrators in New 
York City as part of an ongoing appraisal of the effect of CAI 
as it operates in the school system v}s a whole and In specific 
schools. 

- 6. The record keeping function of the CAI software should be modi- 
fied 30 that the data is stored in such a way as to be readily 
retrievable for answering questions about management of CAI 
operations as well as research and evaluation questions. 

7* If future evaluations are to be conducted evaluators should be 
permitted to set up more rigorous designs so that (1) more con- 
trol can be exercised over the selection of the participants 
In the CAI study and (2) so that less e^ensive alternative 
instructional techniques such as programmed instruction, tutor- 
ing, workbooks, etc., can be evaluated simultaneously with CAI. 
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8* Although CAI requires an expensive installation and its use 
involves some difficult adjustments, the data accumulated 
thus far do not rule out the strong claims initially made for 
CAI as a potentially powerful aid to instruction. In view of 
the initially favorable comparative test results and in view 
of the strong positive motivations that CAI appears to engender, 
experimentation should certainly continue, and effort should 
be concentrated on overcoming weaknesses in the content and 
conditions of use of this particular CAI operation. 
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Table 1 

Kean bWoer of Items, Error Rates, and Latencies during the Months 
of October and Rovem’ter, 1969 for a Random Sample of 101 
Fourth Grace Fupils Grouped According to KAT 
Pre-Test Score 



i 


Total Xo, 
of Iteiris 
CoTTOleted 


Percent: 

Correct 


Percent 

Incorrect 


Percent 

Tirr.e-Outs 


Latency 
Correct Ans. 
(seconds) 


Latency In- 
correct Aus. 
(seconds) 


MAT High 


162.50 


.85 


.10 


.04 


3.88 


5.26 


S = 32 














MAT Average 


134.29 


.79 


,15 


.06; 


■ 4.03 


5.34 ■ 


M = 41 














MAT Low 


124.68 


.71 


.22 


.07 


4.15 


4.71 



S = 28 
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Summary of the Analyses of the KAT Test T}ata Comparing CAI and Non*'CAI J^upils 
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indicates the comparison favored the tion-CAI group. 



